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Abstract — We show how real-number codes can be used to 
compress correlated sources, and establish a new framework for 
distributed lossy source coding, in which we quantize compressed 
sources instead of compressing quantized sources. This change in 
the order of binning and quantization blocks makes it possible 
to model correlation between continuous-valued sources more 
realistically and correct quantization error when the sources are 
completely correlated. The encoding and decoding procedures 
are described in detail, for discrete Fourier transform (DFT) 
codes. Reconstructed signal, in the mean-squared error sense, is 
seen to be better than or close to quantization error level in the 
conventional approach. 

Index Terms — Distributed source coding, real-number codes, 
BCH-DFT codes, channel coding, Wyner-Ziv coding. 



I. Introduction 

The distributed source coding (DSC) deals with compres- 
sion of correlated sources which do not communicate with 
each other [Q~|. Lossless DSC (Slepian-Wolf coding), has been 
realized by different binary channel codes, including LDPC 
[fj] and turbo codes [3|. The Wyner-Ziv coding problem |4j, 
deals with lossy data compression with side information at 
the decoder, under a fidelity criterion. Current approach in 
the DSC of a continuous-valued source is to first convert 
it to a discrete-valued source using quantization, and then 
to apply Slepian-Wolf coding in the binary field. Similarly, 
a practical Wyner-Ziv encoder is realized by cascading a 
quantizer and Slepian-Wolf encoder Q, (6). In other words, 
the quantized source is compressed. There are, hence, source 
coding (or quantization) loss and channel coding (or binning) 
loss. This approach is based on the assumption that there is still 
correlation remaining in the quantized version of correlated 
sources. 

In this paper, we establish a new framework for the Wyner- 
Ziv coding. We propose to first compress the continuous- 
valued source and then quantize it, as opposed to the con- 
ventional approach. The compression is thus in the real field, 
aiming at representing the source with fewer samples. 

To do compression, we generate either syndrome or parity 
samples of the input sequence using a real-number channel 
code, similar to what is done to compress a binary sequence 
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of data using binary channel codes. Then, we quantize these 
syndrome or parity samples and transmit them. There are 
still coding (binning) and quantization losses; however, since 
coding is performed before quantization, error correction is 
in the real field and quantization error can be corrected when 
two sources are completely correlated over a block of code. A 
second and more important advantage of this approach is the 
fact that the correlation channel model can be more realistic, as 
it captures the correlation between continuous-valued sources 
rather than quantized sources. In the conventional approach, 
it is implicitly assumed that quantization of correlated signals 
results in correlated sequences in the discrete domain which 
is not necessarily correct due to nonlinearity of quantization 
operation. In addition, most of previous works assume that 
this correlation, in the binary field, can be modeled by a binary 
symmetric channel (BSC) with a known crossover probability. 
To avoid the loss due to inaccuracy of correlation model, we 
exploit correlation between continuous-valued sources before 
quantization. 

Specifically, we use real BCH-DFT codes [7|, for compres- 
sion in the real field. Owing to the DFT codes, the loss due 
to quantization can be decreased by a factor of k/n for an 
(n,k) DFT code [8 |, [9|. Additionally, if the two sources are 
perfectly correlated over one codevector, reconstruction loss 
vanishes. This is achieved in view of modeling the correlation 
between the two sources in the continuous domain. Finally, 
the proposed scheme seems more suitable for low-delay com- 
munication because using short DFT codes a reconstruction 
error better than quantization error is achievable. 

The rest of this paper is organized as follows. In Section HIl 
we motivate and introduce a new framework for lossy DSC. In 
Section [TTTJ we briefly review encoding and decoding in real 
DFT codes. Then in Section HVl we present the DFT encoder 
and decoder for the proposed system, both in the syndrome and 
parity approaches. These two approaches are also compared 
in this section. Section [V] discusses the simulation results. 
Section IVTl provides our concluding remarks. 

II. Proposed System and Motivations 

We introduce the use of real-number codes in lossy com- 
pression of correlated signals. Specifically, we use DFT codes 
0, a class of real Bose-Chaudhuri-Hocquenghem (BCH) 
codes, to preform compression. Similar to error correction in 
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Fig. 1. The Wyner-Ziv coding using real-number codes. 



finite fields, the basic idea of error correcting codes in the 
real field is to insert redundancy to a message vector of k 
samples to convert it to a codevector of n samples (n > k) 
(T) ■ But unlike that, the insertion of redundancy in the real 
field is performed before quantization and entropy coding. 
The insertion of soft redundancy in the real-number codes has 
advantages over hard redundancy in the binary field. By using 
soft redundancy, one can go beyond quantization error, and 
thus reconstruct continuous-valued signals more accurately. 
This makes real-number codes more suitable than binary codes 
for lossy distributed source coding. 

The proposed system is depicted in Fig. [T] Although it con- 
sists of the same blocks as existing practical Wyner-Ziv coding 
scheme J5), J6), the order of these blocks is changed here. That 
is, we perform Slepian-Wolf coding before quantization. This 
change in the order of the DSC and quantization blocks brings 
some advantages as described in the following. 

• Realistic correlation model: In the existing framework 
for lossy DSC, correlation between two sources is mod- 
eled after quantization, i.e., in the binary domain. More 
precisely, correlation between quantized sources is usu- 
ally modeled as a BSC, mostly with known crossover 
probability. Admittedly though, due to nonlinearity of 
quantization operation, correlation between the quantized 
signals is not known accurately even if it is known 
in the continuous domain. This motivates investigating 
a method that exploits correlation between continuous- 
valued sources to perform DSC. 

• Alleviating quantization error: In lossy data compres- 
sion with side information at the decoder, soft redun- 
dancy, added by DFT codes, can be used to correct both 
quantization errors and (correlation) channel errors. The 
loss due to quantization error thus can be recovered, at 
least partly if not wholly. More precisely, if the two 
sources are exactly the same over a codevector, quanti- 
zation error can be corrected completely. That is, perfect 
reconstruction is achieved over corresponding samples. 
The loss due to quantization error is decreased even if 
correlation is not perfect, i.e., when (correlation) channel 
errors exist. 

• Low-delay communication: If communication is subject 
to low-delay constraints, we cannot use turbo or LDPC 
codes, as their performance is not satisfactory for short 
code length. Whether low-delay requirement exists or not 
depends on the specific applications. However, even in the 
applications that low-delay transmission is not impera- 
tive, it is sometimes useful to consider low-dimensional 
systems for their low computational complexity. 



III. Encoding and Decoding with BCH-DFT Codes 

Real BCH-DFT codes, a subset of complex BCH codes j7J, 
are linear block codes over the real field. Any BCH-DFT code 
satisfies two properties. First, as a DFT code, its parity-check 
matrix is defined based on the DFT matrix. Second, similar 
to other BCH codes, the spectrum of any codevector is zero 
in a block of d — 1 cyclically adjacent components, where d 
is the designed distance of that code [10]. A real BCH-DFT 
codes, in addition, has a generator matrix with real entries, as 
described below. 

A. Encoding 

An (n,k) real BCH-DFT code is defined by its generator 
and parity-check matrices. The generator matrix is given by 
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in which and W„ respectively are the DFT and IDFT 
matrices of size k and n, and S is an n x k matrix with n — k 
zero rows lfTT1 -|fT4|. Particularly, for odd k, S has exactly k 
nonzero elements given as Eoo = 1, = ^n-i,k-i = L 
i = 1 : ATI . 1121 . This guarantees the spectrum of any 
codeword to have n — k consecutive zeros, which is required 
for any BCH code [jl0|. The parity-check matrix H, on the 
other hand, is constructed by using the n — k columns of 
corresponding to the n — k zero rows of X. Therefore, due to 
unitary property of > HG = 0. 

In the rest of this paper, we use the term DFT code in lieu of 
real BCH-DFT code. Besides, we only consider odd numbers 
for k and n; thus, the error correction capability of the code 
is t = L^J =2^. 



B. Decoding 

For decoding, we use the extension of the well-known 
Peterson-Gorenstein-Zierler (PGZ) algorithm to the real field 
[ 1 1 . This algorithm, aimed at detecting, localizing, and es- 
timating errors, works based on the syndrome of error. We 
summarize the main steps of this algorithm, adapted for a 
DFT code of length n, in the following. 

1) Compute vector of syndrome samples 

2) Determine the number of errors v by constructing a 
syndrome matrix and finding its rank 

3) Find coefficients Ai,...,A„ of error-locating polyno- 
mial A(x) — Yl"—iO-~ xXi) whose roots are the inverse 
of error locations 

4) Find the zeros Xf , . . . , X^ 1 of A (a;); the errors are 
then in locations ii, . . . , i v where X\ = a 11 , . . . , X v = 
a 1 " and a — e~ J ~ 

5) Finally, determine error magnitudes by solving a set of 
linear equations whose constants coefficients are powers 
of X t . 

As mentioned, the PGZ algorithm works based on the 
syndrome of error, which is the syndrome of the received 
codevector, neglecting quantization. Let r = y + e be the 
received vector, then 



s — Hr — H(y + e) — He, 



(2) 
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where s — [sx, S2, ■ • ■ , S2t] T is a complex vector of length 
n — k. In practice however, the received vector is distorted by 
quantization (r = y + e, y = y + q) and its syndrome is no 
longer equal to the syndrome of error because 

s = Hr = H(y + q + e) = s q + s e , (3) 

where s q = Hq and s e = He. While the "exact" value 
of errors is determined neglecting quantization, the decoding 
becomes an estimation problem in the presence of quantiza- 
tion. Then, it is imperative to modify the PGZ algorithm to 
detect errors reliably |[T0l - lfT3l . Error detection, localization, 
and also estimation can be largely improved using least squares 
methods Ifl4l . 

C. Performance Compared to Binary Codes 

DFT codes by construction are capable of decreasing quan- 
tization error. When there is no error, an (n, k) DFT code 
brings down the mean-squared error (MSE), below the level 
of quantization error, with a factor of R c = k/n (8), (9). 
This is also shown to be valid for channel errors, as long as 
channel can be modeled as by additive noise. To appreciate 
this, one can consider the generator matrix of a DFT code as 
a tight frame (9); it is known that frames are resilient to any 
additive noise, and tight frames reduce the MSE k/n times 
1 15 1. Hence, DFT codes can result in a MSE even better than 
quantization error level whereas the best possible MSE in a 
binary code is obviously lower-bounded by quantization error 
level. 

IV. Wyner-Ziv Coding Using DFT Codes 

The concept of lossy DSC and Wyner-Ziv coding in the 
real field was described in Section [TTJ In this section, we use 
DFT codes, as a specific means, to do Wyner-Ziv coding in 
the real field. This is accomplished by using DFT codes for 
binning, and transmitting compressed signal, in the form of 
either syndrome or parity samples. 

Let s be a sequence of i.i.d random variables x\x 2 ■ ■ ■ x n , 
and y be a noisy version of x such that yi = Xi+ei, where e, is 
continuous, i.i.d., and independent of X{. Since e is continuous, 
this model precisely captures any variation of x, so it can 
model correlation between x and y accurately. For example, 
the Gaussian, Gaussian Bernoulli-Gaussian, and Gaussian- 
Erasure correlation channels can be modeled using this model 
[16|. These correlation models are practically important in 
video coders that exploit Wyner-Ziv concepts, e.g., when the 
decoder builds side information via extrapolation of previously 
decoded frames or interpolation of key frames Ifl6l . In this 
paper, the virtual correlation channel is assumed to be a 
Bernoulli-Gaussian channel, inserting at most t random errors 
in each codeword; thus, e is a sparse vector. 

A. Syndrome Approach 

1 ) Encoding: Given H, to compress an arbitrary sequence 
of data samples, we multiply it with H to find the corre- 
sponding syndrome samples s x — Hx. The syndrome is then 
quantized (s x = s x + q), and transmitted over a noiseless 
digital communication system, as shown in Fig. [2] Note that 
s x , s x are both complex vectors of length n — k. 
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Fig. 2. The Wyner-Ziv coding using DFT codes: Syndrome approach. 

2) Decoding: The decoder estimates the input sequence 
from the received syndrome and side information y. To this 
end, it needs to evaluate the syndrome of channel (correlation) 
errors. This can be simply done by subtracting the received 
syndrome from syndrome of side information. Then, neglect- 
ing quantization, we obtain, 

S e = S y - S x> ( 4 ) 

and s e can be used to precisely estimate the error vector, as 
described in Section IIII-BI In practice, however, the decoder 
knows s x = s x + q rather than s x . Therefore, only a distorted 
syndrome of error is available, i.e., 

S e = S y - K = S e - q. (5) 

Hence, using the PGZ algorithm, error correction is accom- 
plished based on (O. Note that, having computed the syndrome 
of error, decoding algorithm in DSC using DFT codes is 
exactly the same as that in the channel coding problem. This 
is different from DSC techniques in the binary field which 
usually require a slight modification in the corresponding 
channel coding algorithm to customize for DSC. 

B. Parity Approach 

Syndrome-based Wyner-Ziv coding is straightforward but 
not very efficient because, in a real DFT code, syndrome 
samples are complex numbers. This means that to transmit 
each sample we need to send two real numbers, one for the 
real part and one for the imaginary part. Thus, the compression 
ratio, using an (n, k) DFT code, is 2 (n-k) wnerea s it is -^-^ 
for a similar binary code. This also imposes a constraint on 
the rate of code, i.e., n < 2k or R c > i, since otherwise 
there is no compression. In the sequel, we explore parity-based 
approach to the Wyner-Ziv coding. 

1) Encoding: To compress x, the encoder generates the 
corresponding parity sequence p with n — k samples. The 
parity is then quantized and transmitted, as shown in Fig. [3] 
instead of transmitting the input data. The first step in parity- 
based system is to find the systematic generator matrix, as G 
in (Q~|) is not in the systematic form. Let H be partitioned as 
H = [Hi | Hs], where Hi is a matrix of size n — k x k, 
and H2 is a square matrix of size n — k. Since H2 is a 
Vandermonde matrix, H^ 1 exist and we can write 

H sys =H^H=[P\I 2t ], (6) 

in which P = H^ 1 Hi is an (n — k) x k matrix, and I^t is 
an identity matrix of size 2t. 
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Fig. 3. The Wyner-Ziv coding using DFT codes: Parity approach. 



The systematic generator matrix corresponding to H sys is 
given by 



G 



sys 



h 
-P 



L 2 



H 2 Hi 



Clearly, H sys G sys — 0. It is also easy to check that 



HG SyS — 0. 



(7) 



(8) 



Therefore, we do not need to calculate H sys and the same 
parity-check matrix H can be used for decoding in the parity 
approach. 

An even easier way to come up with systematic generator 

Gi 



matrix is to partition G as 



Go 



where G\ is a square 



matrix of size k. Then, from HG = and the fact that H2 
is invertible one can see G 2 = -H~ X H X G X \ thus, we have 



G = 



Gi ' 




G 2 





H 2 ' 



H 



Gi. 



(9) 



Note that G\ is invertible because using ([T]) any k x k sub- 
matrix of G can be represented as product of a Vandermonde 
matrix and the DFT matrix W/.- This is also proven using a 
different approach in [9 |, where it is shown that any subframe 
of G is a frame and its rank is equal to k. Hence, since G\ 
is invertible, the systematic generator matrix is given by 



G S ys GG X 



(10) 



Again HG sys = because HG = 0. Therefore, the same 
parity-check matrix H can be used for decoding in the parity 
approach. It is also easy to see that G sys is a real matrix. 
The question that remains to be answered is whether G sys 
corresponds to a BCH code? To generate a BCH code, G sys 
must have n — k consecutive zeros in the transform domain. 
W„Gsys = {W n G)G 1 ~ 1 , the Fourier transform of this matrix 
satisfies this condition because W„G, the Fourier transform 
of original matrix, satisfies that. 

Note that, since parity samples, unlike syndrome samples, 
are real numbers, using an (n, k) DFT code a compression 
ratio of -^-^ is achieved. Obviously, a compression ratio of 
is achievable if we use a (2n — k, n) DFT code. 

2) Decoding: A parity decoder estimates the input se- 
quence from the received parity and side information y. 
Similar to the syndrome approach, at the decoder, we need 
to find the syndrome of channel (correlation) errors. To do 
this, we append the parity to the side information and form a 



vector of length n whose syndrome, neglecting quantization, 
is equal to the syndrome of error. That is, 
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hence, 

s z = s e , (12) 
Similarly, when quantization is involved (p = p + q), we get 
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(13) 



(14) 



in which, s q = Hq n . Therefore, we obtain a distorted version 
of error syndrome. In both cases, the rest of the algorithm, 
which is based on the syndrome of error, is similar to that in 
the channel coding problem using DFT codes. 

C. Comparison Between the Two Approaches 

As we saw earlier, using an (n, k) code the compression 
ratio in the syndrome and parity approaches, respectively, 
is 2 (n-k) anc ^ n^-k' Hence, the parity approach is 2k /n = 
2R C > 1 times more efficient than the syndrome approach. 
Conversely, we can find two different codes that result in 
same compression ratio, say ^3^- We know that in the parity 
approach, a (2n — k, n) code can be used for this matter. It is 
also easy to verify that, in the syndrome approach, a code with 
rate R c — results in the same compression. For odd n 
and k, the (n, ^j^) DFT code gives the desired compression 
ratio. Thus, for a given compression ratio the parity approach 
implies a code with smaller rate compared to the code required 
in the syndrome approach. 

V. Simulation Results 

We evaluate the performance of the proposed systems using 
a Gauss-Markov source with zero mean, unit variance, and 
correlation coefficient 0.9; the effective range of the input 
sequences is thus [—4,4]. The sources sequences are binned 
using a (7, 5) DFT code. The compressed vector, either 
syndrome or parity, is then quantized with a 6-bit uniform 
quantizer, and transmitted over a noiseless communication 
media. The correlation channel randomly inserts one error 
(f = 1), generated by a Gaussian distribution. The decoder 
localizes and decodes errors. We compare the MSE between 
transmitted and reconstructed codevectors, to measurers end to 
end distortion. In all simulations, we use 20,000 input frames 
for each channel-error-to-quantization-noise ratio (CEQNR). 
We vary the CEQNR and plot the resulting MSE. The result 
are presented in Fig. [4] and compared against the quantization 
error level in the existing lossy DSC methods. 

It can be observed that the MSE in the syndrome approach 
is lower than quantization error except for a small range of 
CEQNR. Similarly, in the parity approach, the MSE is less 
than quantization error for a wide range of CEQNR. Note 
that in lossy DSC using binary codes, the MSE can be equal 
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Fig. 4. Reconstruction error in the syndrome and parity approaches, using a 
(7, 5) DFT code in Fig. 2, 3. For both schemes, the virtual correlation channel 
inserts one error at each channel error to quantization noise ratio. 



Fig. 5. Relative frequency of correct localization of correlation channel 
error in the syndrome and parity approaches, using a (7, 5) DFT code. 



to quantization error only if the probability of error is zero. 
The performance of both algorithms improves as CEQNR is 
very high. This improvement is due to better error localization, 
since the higher the CEQNR the better the error localization, 
as shown in Fig. [5] and [11 1. At very low CEQNRs, although 
error localization is poor, the MSE is still very low because, 
compared to quantization error, the errors are so small that the 
algorithm may localize and correct some of quantization errors 
instead. Additionally, reconstruction error is always reduced 
with a factor of R c = k/n, in an (n, k) DFT code. 

In terms of compression, the parity approach is 2R C = ^ 
times more efficient than the syndrome approach, as discussed 
earlier in Section IIV-CI Not surprisingly though, the per- 
formance of the parity approach is not as good as that of 
the syndrome approach, because it contains fewer redundant 
samples. On top of that, in this simulation, 1/5 of samples are 
corrupted in the parity approach while this figure is 1/7 for 
the syndrome approach. The parity approach, however, suffers 
from the fact that dynamic range of parity samples, generated 
by ( [Tol l, could be much higher than that of syndrome samples 
as t increases. This implies more precision bits to achieve 
the same accuracy. Finally, it is worth mentioning that when 
data and side information are the same over a block of code, 
reconstruction error becomes zero in both approaches. 

VI. Conclusions 

We have introduced a new framework for distributed lossy 
source coding in general, and Wyner-Ziv coding specifically. 
The idea is to do binning before quantizing the continuous- 
valued signal, as opposed to the conventional approach where 
binning is done after quantization. By doing binning in the 
real field, the virtual correlation channel can be modeled more 
accurately, and quantization error can be corrected when there 
is no error. In the new paradigm, Wyner-Ziv coding is realized 
by cascading a Slepian-Wolf encoder with a quantizer. We 
employ real BCH-DFT codes to do the Slepian-Wolf in the 
real field. At the decoder, by introducing both syndrome- 
based and parity-based systems, we adapt the PGZ decoding 
algorithm accordingly. From simulation results, we conclude 



that our systems, specifically with short codes, can improve the 
reconstruction error, so that they may become viable in real- 
world scenarios, where low-delay communication is required. 
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